Probabilistic Graphical Model
This page contains resources about Probabilistic Graphical Models, Probabilistic Machine Learning and Probabilistic Models, including Latent Variable Models. Bayesian and non-Bayesian approaches can either be used. Subfields and Concepts See Category:Probabilistic Graphical Models for some of its subfields. * Bayesian Networks (directed graphical models) - not '' necessarily following a "Bayesian" approach'' ** Naive Bayes classifier (generative model) *** Bayesian Naive Bayes *** Tree Augmented Naive Bayes ** Logistic Regression (discriminative model) ** Gaussian Bayes Network / Gaussian Belief Net / Directed Gaussian Graphical Model ** Dynamic Bayesian Network *** Hidden Markov Model (HMM) *** Linear Dynamical System / State Space Model **** Kalman filter / Linear Gaussian State Space Model **** Time Series Model ** Deep Belief Network ** Hierarchical Bayesian Model ** Stochastic Computation Graph ** Factor Analyzer ** Auto-Regressive Network / Fully-visible Bayes Network (FVBN) ** Variational Autoencoder (VAE) * Markov Random Fields (undirected graphical models) ** Gibbs Random Field ** Gaussian MRF / Undirected Gaussian Graphical Model ** Lattice Model *** Potts Model *** Ising Model ** Hopfield Network ** Boltzmann Machine *** Restricted Boltzmann Machine **Conditional Random Field **Structural Support Vector Machine **Deep Boltzmann Machine **Associative Markov Network **Maximum Entropy (Maxent) Model **Structural Support Vector Machine (SSVM) / Max Margin Markov Network (M3net) **Factor Graph * Stochastic Models (Stochastic Processes, Random Fields, ...) * Latent Variable Models (i.e. Partially Observed Probabilistic Models) ** Continuous Latent Variable Models *** Factor Analyzer *** Gaussian Process Latent Variable Model (GPLVM) *** Gauss-Markov Process *** State Space Model **** Kalman filter / Linear Gaussian SSM ** Discrete Latent Variable Models *** Latent Dirichlet Allocation *** Hidden Markov Model *** Mixture Model **** Bayesian Model **** Non-Bayesian Model * Mixed Networks (i.e. both deterministic and probabilistic) * Chain Graph / Mixed Graph (i.e. both directed and undirected edges) * Structure Learning ** PC Algorithm ** Network Scoring ** Chow-Liu Trees ** Minimal I-Map ** Bayesian Model Selection ** Annealed Importance Sampling ** Sparsity promoting priors *** L2-regularization / Bayesian Ridge Regression / Gaussian prior *** L1-regularization / Bayesian LASSO / Laplace prior *** Spike and Slab / Bernoulli-Gaussian prior * Inference in graphical models / Probabilistic Inference ** Exact Inference / Exact Marginalization *** Enumeration *** Variable Elimination Algorithm / Bucket Elimination *** Sum-Product Algorithm / Belief Propagation / Sum-Product Message Passing / Factor Graph propagation *** Max-Product Algorithm / Max-Product Belief Propagation / Max-Sum Algorithm *** Conditioning *** Junction Tree Algorithm / Clique Tree Propagation *** Forward-Backward Algorithm (used for HMM) *** Baum-Welch Algorithm (used for HMM) *** Viterbi Algorithm (used for HMM) ** Approximate Inference *** Deterministic / Structural: Variational Bayesian Inference (as Optimization) *** Stochastic: Monte Carlo Inference / Sampling Inference / Particle-based Inference *** Laplace Approximation Online Courses Video Lectures * Probabilistic Graphical Models by Daphne Koller * Machine Learning, Probability and Graphical Models by Sam Roweis - VideoLectures.Net * Graphical Models by Zoubin Ghahramani - VideoLectures.Net * Graphical Models by Cedric Archambeau - VideoLectures.Net * Introduction to Graphical Models for Data Mining by Arindam Banerjee - VideoLectures.Net * Bayesian Learning by Zoubin Ghahramani - VideoLectures.Net * Graphical modelling and Bayesian structural learning by Peter Green - VideoLectures.Net *Graphical Models by Christian Borgelt *Learning Bayesian Networks by Richard E. Neapolitan - VideoLectures.Net *Machine Learning, Probability and Graphical Models by Sam Roweis - VideoLectures.Net *Probabilistic Graphical Models by Sam Roweis - VideoLectures.Net Lecture Notes * Probabilistic Graphical Models by Sargur Srihari * Probabilistic Graphical Models by David Sontag * Probabilistic Graphical Models by Andreas Krause * Probabilistic Graphical Models by Eric Xing * Probabilistic Graphical Models Course by Sargur Srihari * Foundations of Graphical Models by David M. Blei * Probabilistic Models of Discrete Data by David M. Blei * Probabilistic Modelling and Reasoning by Amos Storkey * COS597C: Advanced Methods in Probabilistic Modeling BY David M. Blei * CS228: Probabilistic Graphical Models by Stefano Ermon * CSC 2541: Topics in Machine Learning: Bayesian Methods for Machine Learning by Radford Neal * CSE 515T: Bayesian Methods in Machine Learning by Roman Garnett * CS 281A/Stat 241A: Statistical Learning Theory - Probabilistic Graphical Models by Michael Jordan * Unsupervised Learning by Lester Mackey * Probabilistic and Unsupervised Learning by Maneesh Sahani - Gatsby * Approximate Inference and Learning in Probabilistic Models by Maneesh Sahani - Gatsby * Machine Learning by Kevin Murphy * Topics in multivariate analysis: Probabilistic graphical models * Advanced Statistical Machine Learning by Stefanos Zafeiriou * Statistical Methods in Computer Science by Su-In Lee * Inference in Graphical Models by Sewoong Oh * COS513: Foundations of Probabilistic Modeling * Probabilistic reasoning and statistical inference by Daniel Lassiter Books and Book Chapters * Jordan, M. I. (TBA) An Introduction to Probabilistic Graphical Models. (draft) * Bellot, D. (2016). Learning Probabilistic Graphical Models in R. Packt Publishing. * Pfeffer, A. (2016). Practical probabilistic programming. Manning Publications Co. * Koduvely, H. M. (2015). Learning Bayesian Models with R. ''Packt Publishing. * Theodoridis, S. (2015). ''Machine Learning: A Bayesian and Optimization Perspective. Academic Press. * Hastie, T., Tibshirani, R., & Wainwright, M. (2015). "Chapter 9: Graphs and Model Selection". Statistical learning with sparsity: the lasso and generalizations. CRC Press. * Davidson-Pilon, C. (2015). Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference. Addison-Wesley Professional. * Ankan, A., & Panda, A. (2015). Mastering Probabilistic Graphical Models Using Python. Packt Publishing Ltd. * Nagarajan, R., Scutari, M., & Lèbre, S. (2013). Bayesian Networks in R. Springer, 122, 125-127. * Barber, D. (2012). Bayesian Reasoning and Machine Learning. Cambridge University Press. * Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. MIT Press. * Duda, R. O., Hart, P. E., & Stork, D. G. (2012). Pattern Classification. John Wiley & Sons. * Neal, R. M. (2012). Bayesian learning for neural networks (Vol. 118). Springer Science & Business Media. * Russell, S. J., & Norvig, P. (2010). "Part IV: Uncertain knowledge and reasoning". Artificial Intelligence: A Modern Approach. Prentice Hall. * Alpaydin, E. (2010). "Chapter 16: Graphical Models". Introduction to machine learning. MIT Press. * Koller, D., & Friedman, N. (2009). Probabilistic Graphical Models. MIT Press. * Darwiche, A. (2009). Modeling and reasoning with Bayesian networks. Cambridge University Press. * Borgelt, C., Steinbrecher, M., & Kruse, R. R. (2009). Graphical Models - Representations for Learning, Reasoning and Data Mining. John Wiley & Sons. * Theodoridis, S., Pikrakis, A., Koutroumbas, K., & Cavouras, D. (2008). "Chapter 9: Context-dependent Classification". Pattern Recognition. 4th Ed. Academic Press. * Wainwright, M. J., & Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends® in Machine Learning, 1''(1-2), 1-305. * Bishop, C. M. (2006). "Chapter 8. Graphical Models". ''Pattern Recognition and Machine Learning. Springer. pp. 359–422. * Jordan, M. I. (2003). An Introduction to Probabilistic Graphical Models. * Jordan, M. I., & Sejnowski, T. J. (Ed.). (2001). Graphical models: Foundations of neural computation. MIT Press. * Cowell, R. G., D., A. Philip, L., Steffen L., & Spiegelhalter, D. J. (1999). Probabilistic Networks and Expert Systems. Springer. * Lauritzen, S. L. (1996). Graphical Models. Oxford University Press. * Jensen, F. (1996). An Introduction to Bayesian Networks. Springer. * Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann. * Jordan, M. I. (Ed.). (1998). Learning in graphical models. Kluwer Academic Publishers. Scholarly Articles * Ghahramani, Z. (2015). Probabilistic machine learning and artificial intelligence. Nature, 521(7553), 452-459. * Larrañaga, P., & Moral, S. (2011). Probabilistic graphical models in artificial intelligence. Applied soft computing, 11(2), 1511-1528. * Airoldi, E. M. (2007). Getting Started in Probabilistic Graphical Models. PLoS Computational Biology, 3(12), e252. * Wainwright, M. J., & Jordan, M. I. (2008). Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends® in Machine Learning, 1(1-2), 1-305. * Koller, D., Friedman, N., Getoor, L., & Taskar, B. (2007). 2 Graphical Models in a Nutshell. Statistical Relational Learning, 13. * Silva, R., Scheine, R., Glymour, C., & Spirtes, P. (2006). Learning the structure of linear latent variable models. Journal of Machine Learning Research, 7''(Feb), 191-246. * Frey, B. J., & Jojic, N. (2005). A comparison of algorithms for inference and learning in probabilistic graphical models. ''IEEE Transactions on pattern analysis and machine intelligence, 27(9), 1392-1416. * Jordan, M. I. (2004). Graphical Models. Statistical Science, 140-155. * Jordan, M. I., & Weiss, Y. (2002). Graphical models: Probabilistic inference.The handbook of brain theory and neural networks, 490-496. Tutorials * Graphical Models: Structure Learning by David Heckermann * Graphical Models: Parameter Learning by Zoubin Ghahramani * Heckerman's Bayes Net Learning Tutorial * A Brief Introduction to Graphical Models and Bayesian Networks by Kevin Murphy * An Introduction to Graphical Models by Michael Jordan * Bayesian Modelling in Machine Learning: A Tutorial Review * Bayesian Methods for Machine Learning - NIPS 2004 * Probabilistic Modelling, Machine Learning, and the Information Revolution by Zoubin Ghahramani - 2012 * Graphical Models by Zoubin Ghahramani - MLSS 2012 * Graphical Models Lectures - 2015 Software *Edward: A library for probabilistic modeling, inference, and criticism - Python with TensorFlow *PRMLT - MATLAB Toolbox for the book of PRML by C. Bishop * pmtk3 - Probabilistic Modeling Toolkit for MLPP book by Murphy in Matlab/Octave (3rd edition) * pyprobml - Python code for MLPP book by K. Murphy * BRMLtoolbox - MATLAB and Julia code for the BRML book by D. Barber * PyBRML - Python code for the BRML book by D. Barber * Bayesian Probabilistic Matrix Factorization - MATLAB *Mens X Machina PGM Toolbox - MATLAB *UGM (undirected graphical models) - MATLAB *Module libpgm - Python *Graphical Models Toolkit (GMTK) *Bayesian Modeling and Monte Carlo Methods * SamIam * BNT - Bayes Net Toolbox in MATLAB * libDAI - C++ * OpenGM - C++ * Stan - Python (PyStan) and R (RStan) interfaces * PyMC3 - Python * Infer.NET - Developed by Microsoft Research * OpenBUGS - Bayesian Inference Using Gibbs Sampling See also * Bayesian Machine Learning * Bayesian Nonparametrics * Estimation Theory * Statistical Learning Theory * Probability Theory * Control Theory Other Resources *Comparison of software toolkits *Probabilistic Graphical Models wiki *Easier Plate Notation in Python using Daft - Python *Graphical Models - Notebook Category:Probability and Statistics Category:Machine Learning Category:Probabilistic Graphical Models Category:Graph Theory